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1. Abstract - 

Thl-a-paper -deaeP'ibes sensory substitution systems for space applications^ Physical 
sensors replace missing huSian receptors and feed information to the interpretive centers 
of a different sense. The brain is plastic enough so that, with training, the subject 
localizes the input as if it were received through the missing receptors. 






Astronauts have difficulty feeling objects through space suit gloves because of their 
thickness and because of the 4.3 psi pressure difference. Miniature force sensors on the 
glove palm drive an electrotactlle belt around the waist, thus augmenting the missing 
tactile sensation. 


A proposed teleoperator system with telepresence for a space robot would Incorporate 
teleproprioception and a force sen 3 or/electrotactlle belt sensory substitution system for 
'teletouch. 


2. Introduction 

Sensory substitution is the provision to the brain of information that is usually in one sensory domain 
(e.g. visual information via the eyes and visual system) by means of the receptors, pathways and brain 
projection, integrative and interpretative areas of another sensory system (e.g. "visual" information through 
the skin and somatosensory system). Some examples include sign language for the deaf, Braille for the blind, 
and the various instrumentation approaches to providing sensory information to persons with specific sensory 
losses, such as tactUe vision substitution systems for blind persons. This paper discusses sensory 
substitution and sensory augmentation in relation to space needs: augmented sensation for astronauts wearing 
the bulky gloves required for extravehicular activity and sensory information from 3pace robot hands to the 
teleoperator . 

3. Brain Plasticity as a Basis for Sensory Substitution 

Among the most remarkable capabilities of the central nervous system (CNS) is the ability to compensate 
for losses caused by Injuries. This capacity demonstrates that other brain areas are available to assume 
functions that were previously mediated by the lo3t neural tissue, or that the functions can be mediated by 
the remaining neural tissue. This property reflects the plasticity of the brain. 


Plasticity is the attribute of the central nervous system In which enduring functional changes take 
place. It is one of the two fundamental properties of the nervous system; the other is lt3 excitability, 
which relates to rapid changes leaving no trace in the nervous system. 

Sensory information reaches the brain in the form of nerve Impulses. There Is no doubt that the temporal 
and spatial patterns of nerve impulses provide the basi3 of our sensory perception; the coding of information 
in the form of nerve impulse patterns Is a fundamental concept In neurophysiology and psychology. For 
example, visual information Is sent along the optic nerves in the form of patterns of nerve action 
potentials. The optical images, per se, reach no farther than the retinal receptors. The brain must 
Interpret the nerve impulses as a visual image, after decoding the patterns of afferent impulses. The degree 
of plasticity available in these mechanisms will determine the functional limitations of sensory substitution 
systems. In sensory substitution, plasticity is probably the most critical factor of all the properties of 
the nervous system [13- 

The transducer functions of a set of lost or unavailable receptors can be mediated by artificial 
receptors. For example, in tactile vision substitution 3y3tems the TV camera assumes the role. The optical 
display must be transduced to a form of stimulation that can be handled by the skin receptors, which then 
assume the functional role of relays. The plastic changes do not occur in the skin receptors or pathways, but 
in the CNS [1 ] [2]. 
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H. Soae Examples of Sensory Substitution 

Two widely U3ed sensory substitution methods are Braille and sign language. The first requires very 
little instrumentation and the latter, none at all; however, both accomplish the necessary sensory 
transformation: information usually In one (the lost) sensory domain is transduced to an appropriate display 

for another, intact sensory system. 

(a) In Braille, letters are changed Into raised dots; a code based on a 6-dot matrix was developed by 
Braille to enable blind persons to read letters with the fingertips. The critical factor is that 
this approach allows the blind person to achieve the same conceptual analysis and mental imagery 
from reading with the fingertips as the sighted person achieves by reading print. 

(b) American sign language (ASL) is an incredibly ambitious and successful sensory substitution 

system. It translates information usually in the auditory (high frequency, low parallel input) 
domain into the visual (low frequency, high parallel input) domain. This is accomplished in real 
time as can be noted by watching a TV news program on which a signer is simultaneously translating 
into ASL ("translation" is the appropriate term: Bellugl and Klima [3] consider ASL to differ 

dramatically from English and other spoken languages, with distinct grammatical patterns and its own 
rules of syntax) . 

A number of sensory substitution systems requiring high technology have been or are being developed. 
These Include tactile vision substitution, tactile auditory substitution, and tactile somatosensory 
substitution for insensate hands and feet. 

(a) Tactile vision substitution—Mith a tactile vision substitution system developed in our laboratory, 
the spatial information gathered by a television camera under the subject's control is delivered to 
the skin through an array of vibratory stimulators or electrodes. With training, the blind subjects 
can identify and correctly locate In space complex forms, objects, figures, and faces. Perspective, 
parallax, size constancy, Including looming and zooming and depth cues, are correctly utilized. The 
subjective localization of the information obtained through the television camera is not on the 
skin; it is accurately located in the three-dimensional space in front of the camera, whether the 
skin stimulation matrix is placed on the back, on the abdomen, on the thigh, or changed from one of 
these body locations to another. 

The instrumentation and the research results have been widely reported [1] [*0 [5] [63 [7j. A 
• curriculum ha3 been developed to teach congenitally blind children visual spatial concepts, and is 
being field tested in the United States and Spain. 

(b) Tactile auditory subst itut ion--A comparable tactile auditory substitution system has been developed 
and Is now a commercial product (Tactlcon). Auditory signals picked up by a microphone are divided 
into frequency bands and each of these drives one of 16 eiectrodes on an electrotactile belt worn 
around the wai 3 t, with low tones at one end and high tones at the other [83. 

(c) Tactile somatosensory 3 ubstltutlon--Some ^ears ago, in collaboration with C. C. Collins, the 

feasibility of providing tactile information to leprosy patients with insensate hands was 
explored. A single strain gage was located In each fingertip of a glove worn on one hand, and the 
information was delivered to the skin of the forehead (where sensation was intact) through five 
electrotactile stimulators. Within a few hours of training, it was possible to Locate the sensation 

on the fingertips, and it was possible to identify various textures [53- As with the tactile vision 

substitution system, correct subjective localization (in this case to the fingertips) required 
active control of movement by the subject. 

The success with the leprosy patient study led to the exploration of other applications. Funded 
studies are now underway to explore the application of this approach to patients with insensate feet 
due to diabetes, and to space suit gloves and space robots. 

5. Space Suit Glove Requirements 

The need for human protection against the space environment began in the Gemini Program of the 19603. 
Since then, manned space flight has progressed through the Apollo and Skylab eras, is currently in the Shuttle 
era, and planning for the Space Station era. Throughout these eras, the space suit glove ha3 evolved into a 
complicated piece of the extravehicular mobility unit (EMU) and has improved greatly in areas of mobility and 
dexterity. The EMU i 3 essentially an anthropomorphic enclosure for which the human can operate in the space 
environment and is shown in Fig. 1 for the Space shuttle era. Extravehicular activity (EVA) has become a 
much-needed resource In space operations and problems with human performance (in ado it Ion to those of a lack 
of EVA manhours available) are still apparent. In the Space Station era, more than 2000 EVA manhours may be 

needed to perform construction, assembly, servicing and maintenance activities. Many tasks for the astronauts 

have been structured around the existing glove capabilities; what activities and tasks could be accomplished 
with an optimal space suit glove? A perceived optimal glove is shown in Fig. 2. 
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Fig. 1 The extravehicular mobility unit (F.MU) la an anthropomorphic enclosure that assists human mobility an. 
dexterity In space. 
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Optimised EVA Glove. 




Fop the future Space Station, 15 generic EVA activities have been defined as references for EVA system 
design. These Include [9]« 

1. Alignment of transmitter and receiver elements 

2. Deployment/retraction of solar arrays 

3. Truss structure construction 

4. Satellite servicing 

5. Large module manipulation 

6. Small module manipulation 

7. Large mirror construction 

8. Consumable recharge via module transport 

9- Orbit launch operations 

10. Satellite operations 

11. Space Station radiator construction (from STS) 

12. Space Station radiator construction (from Space Station) 

13. STS-supported large module manipulation 

14. STS-supported truss construction/deployment 

15. EVA rescue. 

Many of these contain tasks and activities which require fine control and manipulation. Installation of 
hardware Including assembly, replacement of orbital replacement units (ORUa) contingency maintenance and 
repair, transfer of equipment In and out of pressurized modules, routine support servicing and handling or 
fluids, equipment stowage, and platform support represent a few of these tasks. The EVA gloves provide the 
major and sometimes only Interface between the astronaut and the work being performed and thus must provide a 
balance of mobility, tactlllty comfort, and protection from the workplace hazards. 


6 . Space Suit Clove Problems 

A problem encountered by astronauts during extravehicular activity is that they have trouble feeling 
objects through their space suit gloves. The glove Is made of several layers of plastic and fabric. The 
plastic prevents air leakage. The cloth provides strength so the plastic will not burst. The cloth also 
provides thermal Insulation and protection from micrometeoroids. It is difficult to feel objects through 
these layers or through the thick silicone rubber fingertips. 


Another even larger problem Is that of the astronaut fatigue from work required to move the glove from 
its neutral position due to the difference between the space suit and space environment pressures. The 

pressure of the space suit for the Shuttle Is 4.3 psi and planned at 8.3 pst for the Space Station. In 

addition, more radiation and micrometeoroid/debr is protection will be required. All of these factors Inhibit 
the design of a flexible and dexterous glove with good sensory capabilities. Answers to both of these 
problems and many more being addressed by NASA would greatly enhance the performance of the astronaut on 
EVA. Current training practices have allowed substitution of perception for the lack of tactlllty. 
Enhancement of the tactile sensory perception may reduce the fatigue problem, Increase the EVA capability to 


finer motor control, allow the enhancement of the glove design without detriment to the tactlllty, reduce the 
astronaut reliance on visual feedback, and thus reduce training time to learn certain tasks. The pressure 
within the glove causes two major effects. The first i3 stiffness of the glove itself reducing freedom of 
position. This restricts the movement of the glove without a great deal of work. Over a large period or time 
(such as the standard EVA of six hours), the hand Is subject to extreme fatigue. Some of this fatigue is due 
to overgripping to ensure contact. Providing the tactile feedback will give the 3ense of contact and reduce 
overgrlpping thereby reducing the fatigue. 


Second, the present air pressure difference of 4.3 psi causes the glove to balloon out. This reduces the 
tactlllty between handholds and tools. The resulting tangential Torces on the surface or the glove make it 
difficult to perceive normal forces through the glove. An analogous situation Is trying to fee] a pehble 
through a bicycle tire. If the tire i3 flat. It 13 easy to feel the pebble through it. If the tire is 
pressurized, it is very difficult. The proposed space suits would have a 8.3 psi pressure difference, whicn 
would make the problem much worse. 

The result of this problem is that the astronaut cannot feel when a tool Is starting to 3lip. He 
overcompensates for tni 3 oy applying a higher than required force to ensure adequate grasp. Hi3 muscles tire 
quickly and he becomes fatigued much sooner. Force Is again used to compensate for thi3 to assure contact. 
To reduce this ballooning effect, which also reduces the astronaut'3 capability to grasp an object, NASA has 
trlgd hard palms and other palm restraints. The restraint is necessary to enable adequate bending of the 
metacarpal joint. Solutions to this problem have met with mixed success during the astronaut evaluations. 
Tactile sensors on the restraint may make such a design feasible In terms of tactlllty and interfacing with 
tools, handholds, and other objects. 


A 3 Or. George Nelson reports of his Solar Max Repair activity In space, glove limitations are minimal for 
gross motor control but are almost Inhibltive for fine motor control. His projection indicates that 201 
dexterity is lost when handling objects of one inch in diameter and 501 for objects of less than one inch. 
For objects of millimeter size, the glove permits almost no fine orator control, due to lack of flexibility in 
conducting more detailed task 3 . Dr. Nelson recommends increased tactlllty especially in the areas of the 
fingertips and the full length of the index finger and thumb. 



Loss of tactllity In the space suit glove has been estimated to cause more detriment to EVA performance 
than is recognized by the crewmembers. Much of the loss of tactllity Is compensated by visual perception of 
the task during the many hundreds of hours of training. Providing tactile feedback would relieve the need for 
such a substitute and possibly reduce the amount of training to perform a specific task. 

NASA has looked at mainly passive means in which to Increase the tactile feedback to the astronaut 
including movable pins, enhanced fingertip tactile pads, glove/hand adhesion, and removable fingertips to 
expose a less bulky, less protected finger. One concept is shown In Fig. 3. Other means such as sensory 
substitution have also been recommended for feeding other senses such as vision or hearing. The concept 
discussed here Is active tactile sensory perception Inducing a simple relocation of the pressure to the 
abdomen or forearm. Thus the other senses of sight and sound are not overburdened. 
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Fig. 3 Thermally Insulated glove contains short, closely-spaced elastomeric pins that Insulate without 
Impairing flexibility, 

7. Space Suit Clove Sensory Substitution 

Based on sensory substitution mentioned above and reported in detail elsewhere, we proposed a study of 
tactile sensory substitution for space suit gloves to increase the performance of extravehicular astronauts by 
Increasing tactile sensation. The working hypothesis Is that sensory Information gathered by the active 
control of hand movement would be subjectively located In the hand, even though the information arrives at the 
body at another location (e.g., skin of the abdomen or arm). 

We have built a sensory substitution system where the Torce sensors are on the outside of the glove and 
are exposed directly to the grasped object. They are located at the points of maximal information, such as 
the inside of the thumb and first two finger s. We determined these locations by observing wear patterns of 
used space suit gloves and by grasping" various tools” using space suit gloves! 

A problem in attaching the Torce transducers is that as the hand is closed, deep wrinkles form on the 
outer fabric layer of the glove. A small transducer attached to the fabric surface might shift into the 
pocket of a wrinkle and not detect the surface force at all. To overcome this problem we placed 0.8-mm thick 
11 -mm diameter metal disks at each location. These had four 1-mm holes at 4 peripheral locations. We used 
nylon flshllne to sew the disks loosely to the fabric. This permitted wrinkles to form but kept the dl3k3 
parallel to the surface. 

Cons^^nt-a -■‘v. the selection of a force transducer are that It should be small, have low power 
consumption, have a wide temperature range, and be rugged. Possible transducer types Include strain gages. 
Inductance transducers, optical transducers, piezoelectric transducers, conductive elastomers, tlme-of-f 1 ight 
transducers, and capacitive transducers. The smallest commercial transducer we found is the Model 105 
pressure transducer: 80 psi, 350 ohm from Precision Measurement Company, Box 7676, Ann Arbor, MI 48107. It 
is 0.28 mm thick and 2.6 mm in diameter. We enclosed the 3 fragile wires from the t-arm metal strain gage 
transducer in heat shrink tubing. Using Dow Corning No. 891 Medical Adhesive Silicone Type A, we glued the 
transducer to the disk and formed a rounded surface of adhesive over the top of the transducer. Thus the 
rubbery adhesive transmitted forces to the diaphragm of the pressure transducer. We routed wires under the 
outer fabric layer from the palm to the wri3t connector. 

We constructed our own electronics to drive a commercial electrotactlle system. Because the transducer 
has only one strain gage element, we added three resistors to complete a Wheatstone bridge. An operational 
amplifier amplified the small signal Trom the bridge to a large signal suitable for driving the electrotactlle 
system. Potentiometers to adjust offset and gain were required. 

We purchased a Tactlcon 1600 electrotactlle sensory aid for the deaf. It has a microphone Input, divides 
speech into frequency bands, and delivers electrical stimuli to the skin through 16 gold-plated 5-mra diameter 
electrodes. A belt around the waist positions the electrodes over the abdomen and receives power from a 
battery pack and electronics located in a box clipped to the user’s normal belt. We removed the speech system 
and fed the drivers from our amplifiers. 
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The system worked well tn that each user could establish cause and effect between pressure on a specific 
transducer and electrical stimulation at a particular electrode. Unfortunately our original estlaate that 80 
pal transducers would be satisfactory was wrong. Firm grasp of tools* such as screwdriver handles 
overstressed the transducer diaphragms beyond the yield strength, resulting In permanent damage. Thus we are 
restricted to light pressures. We have ordered 1000 psl transducers and will test those under aore rigorous 
conditions. 

We expect that a period of learning will be required for optimal Interpretation of this alternative 
sensory system. We plan an evaluation by skilled astronauts performing typical tasks in a pressurized glove 
box after an 8-hour learning session. 

8. Space Robots 

NASA has proposed a space robot that would perform tasks in space under the control of an astronaut in a 
spacecraft or by ground controllers. The astronaut would control the robot In a master-slave relationship 
called teleoperation. The astronaut would place his hands and arms in controlling gloves and mechanical 
arms. When the astronaut would move, the robot would exactly follow him. 

Figure ** shows an additional system, called telepresence, which would provide sensation to the 
astronaut. When the robot's hand would close on an object, the astronaut's hand inside the controlling glove 
would feel the same mechanical resistance through proprioception. This telepropr ioception [10] would be 
accomplished by actuators In the astronaut's controlling glove which would be slaved to the actuator in the 
robot's hand and increase the force that resists controlling glove closure. The astronaut would receive 
direct position reedback of robot Joint angles from his own Joint receptors. He would receive direct force 
feedback of robot forces from his own muscle and tendon receptors. Telepropr Ioception would provide feedback 
about the large forces resulting from firm grasp, but not the small forces associated with slip. Figure 5 
shows a one-degree-of-freedom 3y3tem that would provide both teleoperation and teleproprioception. 

None of the telerobotic systems described above Include teletouch C 1 0 3 . The human palmar skin contains a 
variety of mechanoreceptors to sense light touch and nociceptors to sense the pain that results from high 
pressures. Thus when the robot 3 en 3 es objects at different locations within the palm, tnls Information should 
be sensed and transmitted to the astronaut. If an object is slipping from the robot's grasp, the astronaut 
should receive the same sensations as if the object were slipping from his grasp. 

Thus the robot's palm should be covered with many force sensors. These should detect small forces 
necessary to detect slip. They should be capable of withstanding large overloads as when the robot grasps 
tools firmly. Most transducers nave a deformable element such as a spring. Most transducers are designed to 
operate to an appreciable fraction of their yield strength to give a large output and thus they overload 
easily. What Is needed Is an element that deforms easily to produce a sensitive range, but then hits a 
mechanical stop to be able to withstand high forces during overload. It will be a challenge to develop such 
transducers in miniature form. 


If the astronaut grasped a sharp pointed object, his skin would indent. The Ideal teletouch system would 
also Indent his skir.. An array of solenoids or air bladders might accomplish this goal, but It would be 
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develop, the next best solution would be a sensory substitution system. Information from touch sensors on the 
'"obot palm would drive stimulators on the palm. These might be vlbrotactlle arrays located In similar palmar 
areas within the astronaut's controlling glove. Electrotactl le stimulators on the palm are not practical 
because the thick 3 kln results tn painful stimulation. But electrotact 1 le stimulators on a belt around the 
waist are practical and could be used In a successful system. 


TELEOPERATION 
I Manipulator 
I position control 


TELEPROPRIOCEPTION 
Joint position and tonq ue 




TELETOUCH 

Obfect contour and texture 


TELEPRESENCE 

Operator 

sensation 


TELEVISION 

TV display o 1 manipulator 



(Other sensations Including 
temperature, sound, etc.) 


Fig. 4 Different elements of a teleoperator system showing the separation of the proprioceptive and tactile 
sensory feedback. 
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Manipulator 
finger joint 



Fig. 5 Block diagram of a one degree-of-freedom force-reflecting control scheme providing teleoperation and 
telepropr iocept ion . 

For a robot to be useful in space, it should be anthropomorphic. Its hand should resemble our hand, so 
that it can perform the same tasks the astronaut can. Also if the robot falls, an astronaut must perform the 
robot's tasks. Several groups have developed anthrcpomorphlc-1 ike hands. None has developed 

teleproprioception for the hand. None has developed teletouch for the hand. It will be a challenge to 
implement telepresence In the limited space available within the normal boundaries of a hand. 
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